Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Installing required libraries¶
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
Importing necessary libraries¶
import os
import numpy as np # Importing numpy for Matrix Operations
import pandas as pd # Importing pandas to read CSV files
import matplotlib.pyplot as plt # Importting matplotlib for Plotting and visualizing images
import math # Importing math module to perform mathematical operations
import cv2 # Importing openCV for image processing
import seaborn as sns
import random # Importing seaborn to plot graphs
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras.callbacks import EarlyStopping #regularization method to prevent the overfitting
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras import backend
# Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score, confusion_matrix
# Display images using OpenCV
from google.colab.patches import cv2_imshow # Importing cv2_imshow from google.patches to display images
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
Loading the dataset¶
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Load the image file of the dataset
images = np.load('/content/drive/MyDrive/AIMLColab/Intro to Computer Vision/CNN-Project1/images.npy')
# Load the labels file of the dataset
labels = pd.read_csv('/content/drive/MyDrive/AIMLColab/Intro to Computer Vision/CNN-Project1/Labels.csv')
Data Overview¶
Understand the shape of the dataset¶
print(images.shape)
print(labels.shape)
(4750, 128, 128, 3) (4750, 1)
There are 4750 RGB images of shape 128 x 128 X 3, each image having 3 channels.
Plotting images using OpenCV and matplotlib¶
cv2_imshow(images[10])
plt.imshow(images[15])
<matplotlib.image.AxesImage at 0x7b1d7b2b84d0>
- We can observe that the images are being shown in different colors when plotted with openCV and matplotlib as OpenCV reads images in BGR format and this shows that the given numpy arrays were generated from the original images using OpenCV.
- Now we will convert these BGR images to RGB images so we could interpret them easily.
Exploratory Data Analysis¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you understand the data better.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
def plot_images(images,labels):
num_classes=10 # Number of Classes
categories=np.unique(labels)
keys=dict(labels['Label']) # Obtaing the unique classes from y_train
rows = 3 # Defining number of rows=3
cols = 4 # Defining number of columns=4
fig = plt.figure(figsize=(10, 8)) # Defining the figure size to 10x8
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
ax.imshow(images[random_index, :]) # Plotting the image
ax.set_title(keys[random_index])
plt.show()
plot_images(images,labels)
Checking for data imbalance¶
sns.countplot(labels['Label'])
plt.xticks(rotation='vertical')
(array([ 0., 100., 200., 300., 400., 500., 600., 700.]), [Text(0.0, 0, '0'), Text(100.0, 0, '100'), Text(200.0, 0, '200'), Text(300.0, 0, '300'), Text(400.0, 0, '400'), Text(500.0, 0, '500'), Text(600.0, 0, '600'), Text(700.0, 0, '700')])
- As you can see from the above plot, the dataset is quite imbalanced.
- There are about
- Shepered Purse, Common Wheat, Maize and Black-grass have around 200 images
- Sugar beet and charlock have around 400 images
- Cleavers around 300 and rest os plant seeds above 400 images
Data Pre-Processing¶
Convert the BGR images to RGB images.¶
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)
Resize the images¶
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
images_decreased=[]
height = 64
width = 64
dimensions = (width, height)
for i in range(len(images)):
images_decreased.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))
plt.imshow(images_decreased[10])
<matplotlib.image.AxesImage at 0x7b1d042af2d0>
Visualizing images using Gaussian Blur¶
# Applying Gaussian Blur to denoise the images
images_gb=[]
for i in range(len(images)):
images_gb.append(cv2.GaussianBlur(images[i], ksize =(3,3),sigmaX = 0))
plt.imshow(images_gb[10])
<matplotlib.image.AxesImage at 0x7b1d043f7b10>
Data Preparation for Modeling¶
- Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
- You'll have to encode categorical features and scale the pixel values.
- You will build a model using the train data and then check its performance
Split the dataset
- As we have less images in our dataset, we will only use 10% of our data for testing, 10% of our data for validation and 80% of our data for training.
- We are using the train_test_split() function from scikit-learn. Here, we split the dataset into three parts, train,test and validation.
from sklearn.model_selection import train_test_split
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images),labels , test_size=0.1, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.1, random_state=42,stratify=y_temp)
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
(3847, 128, 128, 3) (3847, 1) (428, 128, 128, 3) (428, 1) (475, 128, 128, 3) (475, 1)
Encode the target labels¶
- Convert labels from names to one hot vectors
- Normalizing the values
# Convert labels from names to one hot vectors.
# Labelbinarizer works similar to onehotencoder
from sklearn.preprocessing import LabelBinarizer
enc = LabelBinarizer()
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)
Data Normalization¶
Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0
Model Building¶
Let's create a CNN model sequentially, where we will be adding the layers one after another.
First, we need to clear the previous model's history from the session even if a single model can run multiple times on the same data.
In Keras, we need a special command to clear the model's history, otherwise the previous model history remains in the backend.
Also, let's fix the seed again after clearing the backend.
Let's set the seed for random number generators in Numpy, the Random library in Python, and in TensorFlow to be able to reproduce the same results every time we run the code.
Model - 1¶
- Kernel_Size: The kernel size here refers to the widthxheight of the filter mask. The kernel_size must be an odd integer as well. Typical values for kernel_size include: (1, 1) , (3, 3) , (5, 5) , (7, 7)
- Padding: The padding type is called SAME because the output size is the same as the input size(when stride=1). Using 'SAME' ensures that the filter is applied to all the elements of the input. Normally, padding is set to "SAME" while training the model. Output size is mathematically convenient for further computation.
- MaxPool2D: Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually used after a convolutional layer.
- Flatten: Flattening is converting the data into a 1-dimensional array for giving them as input to the next layer. We flatten the output of the convolutional layers to create a single long feature vector. And it is connected to the final classification model, which is called a fully-connected layer.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
Now, let's build a CNN Model with the following 2 main parts -
- The Feature Extraction layers which are comprised of convolutional and pooling layers.
- The Fully Connected classification layers for prediction.
# Intializing a sequential model
model1 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding = 'same'))
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model1.add(Dense(100, activation='relu'))
model1.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model1.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 128, 128, 64) 1792
max_pooling2d (MaxPooling2 (None, 64, 64, 64) 0
D)
conv2d_1 (Conv2D) (None, 64, 64, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 32, 32, 32) 0
g2D)
conv2d_2 (Conv2D) (None, 32, 32, 32) 9248
max_pooling2d_2 (MaxPoolin (None, 16, 16, 32) 0
g2D)
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 100) 819300
dropout (Dropout) (None, 100) 0
dense_1 (Dense) (None, 12) 1212
=================================================================
Total params: 850016 (3.24 MB)
Trainable params: 850016 (3.24 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
- There are 850016 trainable parameter
Fitting the model on the train data¶
Early stopping is a technique similar to cross-validation where a part of training data is kept as the validation data. When the performance of the validation data starts worsening, the model will immediately stop the training.
- Monitor: Quantity to be monitored.
- Mode: One of {"auto", "min", "max"}. In min mode, training will stop when the quantity monitored has stopped decreasing; in "max" mode it will stop when the quantity monitored has stopped increasing; in "auto" mode, the direction is automatically inferred from the name of the monitored quantity.
- Patience: Number of epochs with no improvement after which training will be stopped.
ModelCheckpoint callback is used in conjunction with training using model. fit() to save a model or weights (in a checkpoint file) at some interval, so the model or weights can be loaded later to continue the training from the state saved.
# Epochs
epochs = 30
# Batch size
batch_size = 64
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.keras', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model1_history = model1.fit(
X_train_normalized, y_train_encoded,
epochs=epochs,
batch_size=batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=2,callbacks=[es, mc]
)
Epoch 1/30 Epoch 1: val_accuracy improved from -inf to 0.36215, saving model to best_model.keras 61/61 - 5s - loss: 2.3564 - accuracy: 0.1843 - val_loss: 2.0248 - val_accuracy: 0.3621 - 5s/epoch - 77ms/step Epoch 2/30 Epoch 2: val_accuracy improved from 0.36215 to 0.53271, saving model to best_model.keras 61/61 - 2s - loss: 1.8133 - accuracy: 0.3933 - val_loss: 1.4986 - val_accuracy: 0.5327 - 2s/epoch - 38ms/step Epoch 3/30 Epoch 3: val_accuracy improved from 0.53271 to 0.57944, saving model to best_model.keras 61/61 - 2s - loss: 1.4142 - accuracy: 0.5157 - val_loss: 1.2976 - val_accuracy: 0.5794 - 2s/epoch - 37ms/step Epoch 4/30 Epoch 4: val_accuracy improved from 0.57944 to 0.62383, saving model to best_model.keras 61/61 - 2s - loss: 1.2407 - accuracy: 0.5706 - val_loss: 1.1350 - val_accuracy: 0.6238 - 2s/epoch - 34ms/step Epoch 5/30 Epoch 5: val_accuracy improved from 0.62383 to 0.64720, saving model to best_model.keras 61/61 - 2s - loss: 1.1624 - accuracy: 0.5963 - val_loss: 1.0111 - val_accuracy: 0.6472 - 2s/epoch - 35ms/step Epoch 6/30 Epoch 6: val_accuracy improved from 0.64720 to 0.69626, saving model to best_model.keras 61/61 - 2s - loss: 0.9931 - accuracy: 0.6616 - val_loss: 0.9124 - val_accuracy: 0.6963 - 2s/epoch - 36ms/step Epoch 7/30 Epoch 7: val_accuracy improved from 0.69626 to 0.71028, saving model to best_model.keras 61/61 - 2s - loss: 0.8858 - accuracy: 0.6899 - val_loss: 0.8791 - val_accuracy: 0.7103 - 2s/epoch - 36ms/step Epoch 8/30 Epoch 8: val_accuracy improved from 0.71028 to 0.74065, saving model to best_model.keras 61/61 - 2s - loss: 0.8031 - accuracy: 0.7195 - val_loss: 0.8117 - val_accuracy: 0.7407 - 2s/epoch - 39ms/step Epoch 9/30 Epoch 9: val_accuracy did not improve from 0.74065 61/61 - 2s - loss: 0.7098 - accuracy: 0.7544 - val_loss: 0.8539 - val_accuracy: 0.7196 - 2s/epoch - 35ms/step Epoch 10/30 Epoch 10: val_accuracy improved from 0.74065 to 0.76168, saving model to best_model.keras 61/61 - 2s - loss: 0.6660 - accuracy: 0.7710 - val_loss: 0.7491 - val_accuracy: 0.7617 - 2s/epoch - 35ms/step Epoch 11/30 Epoch 11: val_accuracy did not improve from 0.76168 61/61 - 2s - loss: 0.5759 - accuracy: 0.7998 - val_loss: 0.7375 - val_accuracy: 0.7617 - 2s/epoch - 34ms/step Epoch 12/30 Epoch 12: val_accuracy improved from 0.76168 to 0.77103, saving model to best_model.keras 61/61 - 2s - loss: 0.5339 - accuracy: 0.8160 - val_loss: 0.7066 - val_accuracy: 0.7710 - 2s/epoch - 35ms/step Epoch 13/30 Epoch 13: val_accuracy did not improve from 0.77103 61/61 - 2s - loss: 0.4866 - accuracy: 0.8284 - val_loss: 0.7725 - val_accuracy: 0.7687 - 2s/epoch - 33ms/step Epoch 14/30 Epoch 14: val_accuracy did not improve from 0.77103 61/61 - 2s - loss: 0.4449 - accuracy: 0.8407 - val_loss: 0.9829 - val_accuracy: 0.7056 - 2s/epoch - 36ms/step Epoch 15/30 Epoch 15: val_accuracy did not improve from 0.77103 61/61 - 2s - loss: 0.4531 - accuracy: 0.8375 - val_loss: 0.7575 - val_accuracy: 0.7710 - 2s/epoch - 37ms/step Epoch 16/30 Epoch 16: val_accuracy improved from 0.77103 to 0.78972, saving model to best_model.keras 61/61 - 2s - loss: 0.3688 - accuracy: 0.8679 - val_loss: 0.7811 - val_accuracy: 0.7897 - 2s/epoch - 34ms/step Epoch 17/30 Epoch 17: val_accuracy did not improve from 0.78972 61/61 - 2s - loss: 0.3703 - accuracy: 0.8677 - val_loss: 0.8301 - val_accuracy: 0.7804 - 2s/epoch - 34ms/step Epoch 17: early stopping
Model Evolution¶
plt.plot(model1_history.history['accuracy'])
plt.plot(model1_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- Accuracy of the model is showing good progress, however Train and Validation data shows diverging after certain epoch
plt.plot(model1_history.history['loss'])
plt.plot(model1_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- Loss predectionis is reducing as with number of Epoch, but again Test and Validation is data is diverging
Observations:
- We can see from the above plot that the training accuracy and validation accuracy was good, same in case with loss predection too.
- The model seems to overfit on the data.
Generating the predictions using test data¶
# Here we would get the output as probablities for each category
model1_y_pred=model1.predict(X_test_normalized)
model1_y_pred_classes_ln = np.argmax(model1_y_pred,axis=1)
model1_y_test = np.argmax(y_test_encoded, axis=1)
15/15 [==============================] - 0s 8ms/step
# Test Accuracy
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(model1_y_test, model1_y_pred_classes_ln)
0.7705263157894737
- Model-1 provides an accuracy of 77%
Plotting the Confusion Matrix¶
- The Confusion matrix is also defined as an inbuilt function in the TensorFlow module, so we can use that for evaluating the classification model.
- The Confusion matrix expects categorical data as input. However, y_test_encoded is an encoded value, whereas y_pred has probabilities. So,we must retrieve the categorical values from the encoded values.
- We will use the
argmax()function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes.
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(model1_y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Observations
- We observe that some of the classes are not predicted correctly.
- We can also observe that classes 0, 4 and 6 are mostly misclassified.
- In comparison to the rest, other classeds are well classified.
Model-2¶
Data Augumentation¶
As we can see, our initial model appears to overfit. Therefore we'll try to address this problem with data augmentation to check if we can improve the model's performance.
In most of the real-world case studies, it is challenging to acquire a large number of images and then train CNNs. To overcome this problem, one approach we might consider is Data Augmentation. CNNs have the property of translational invariance, which means they can recognise an object even if its appearance shifts translationally in some way.
Taking this attribute into account, we can augment the images using the techniques listed below -
1. Horizontal Flip (should be set to True/False)
2. Vertical Flip (should be set to True/False)
3. Height Shift (should be between 0 and 1)
4. Width Shift (should be between 0 and 1)
5. Rotation (should be between 0 and 180)
6. Shear (should be between 0 and 1)
7. Zoom (should be between 0 and 1) etc.
Remember, data augmentation should not be used in the validation/test data set.
Let's look at some examples of our augmented training data.
This is helpful for understanding the extent to which data is being manipulated prior to training, and can be compared with how the raw data looks prior to data augmentation.
# All images to be rescaled by 1/255.
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Intializing a sequential model
model2 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension images
model2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding = 'same'))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())
# Adding a fully connected dense layer with 100 neurons
model2.add(Dense(100, activation='relu'))
model2.add(Dropout(0.3))
model2.add(Dense(12, activation='softmax'))
opt=Adam()
# Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# The model is built automatically when it is compiled, or when it is first used.
model2.summary() # This line should now work without error.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 128, 128, 64) 1792
max_pooling2d (MaxPooling2 (None, 64, 64, 64) 0
D)
conv2d_1 (Conv2D) (None, 64, 64, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 32, 32, 32) 0
g2D)
conv2d_2 (Conv2D) (None, 32, 32, 32) 9248
max_pooling2d_2 (MaxPoolin (None, 16, 16, 32) 0
g2D)
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 100) 819300
dropout (Dropout) (None, 100) 0
dense_1 (Dense) (None, 12) 1212
=================================================================
Total params: 850016 (3.24 MB)
Trainable params: 850016 (3.24 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Epochs
epochs = 30
# Batch size
batch_size = 64
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.keras', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model2_history = model2.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1, callbacks=[es, mc])
Epoch 1/30 60/60 [==============================] - ETA: 0s - loss: 2.2371 - accuracy: 0.2358 Epoch 1: val_accuracy improved from -inf to 0.40421, saving model to best_model.keras 60/60 [==============================] - 17s 247ms/step - loss: 2.2371 - accuracy: 0.2358 - val_loss: 1.8349 - val_accuracy: 0.4042 Epoch 2/30 60/60 [==============================] - ETA: 0s - loss: 1.7146 - accuracy: 0.4071 Epoch 2: val_accuracy improved from 0.40421 to 0.55607, saving model to best_model.keras 60/60 [==============================] - 15s 243ms/step - loss: 1.7146 - accuracy: 0.4071 - val_loss: 1.4137 - val_accuracy: 0.5561 Epoch 3/30 60/60 [==============================] - ETA: 0s - loss: 1.3858 - accuracy: 0.5133 Epoch 3: val_accuracy improved from 0.55607 to 0.62617, saving model to best_model.keras 60/60 [==============================] - 15s 244ms/step - loss: 1.3858 - accuracy: 0.5133 - val_loss: 1.1326 - val_accuracy: 0.6262 Epoch 4/30 60/60 [==============================] - ETA: 0s - loss: 1.2367 - accuracy: 0.5752 Epoch 4: val_accuracy improved from 0.62617 to 0.65888, saving model to best_model.keras 60/60 [==============================] - 15s 252ms/step - loss: 1.2367 - accuracy: 0.5752 - val_loss: 1.0467 - val_accuracy: 0.6589 Epoch 5/30 60/60 [==============================] - ETA: 0s - loss: 1.1171 - accuracy: 0.6212 Epoch 5: val_accuracy improved from 0.65888 to 0.68224, saving model to best_model.keras 60/60 [==============================] - 15s 244ms/step - loss: 1.1171 - accuracy: 0.6212 - val_loss: 0.9814 - val_accuracy: 0.6822 Epoch 6/30 60/60 [==============================] - ETA: 0s - loss: 1.0459 - accuracy: 0.6379 Epoch 6: val_accuracy improved from 0.68224 to 0.71495, saving model to best_model.keras 60/60 [==============================] - 15s 244ms/step - loss: 1.0459 - accuracy: 0.6379 - val_loss: 0.8912 - val_accuracy: 0.7150 Epoch 7/30 60/60 [==============================] - ETA: 0s - loss: 0.9536 - accuracy: 0.6751 Epoch 7: val_accuracy did not improve from 0.71495 60/60 [==============================] - 15s 243ms/step - loss: 0.9536 - accuracy: 0.6751 - val_loss: 0.8750 - val_accuracy: 0.6939 Epoch 8/30 60/60 [==============================] - ETA: 0s - loss: 0.8621 - accuracy: 0.7029 Epoch 8: val_accuracy improved from 0.71495 to 0.74299, saving model to best_model.keras 60/60 [==============================] - 15s 243ms/step - loss: 0.8621 - accuracy: 0.7029 - val_loss: 0.7935 - val_accuracy: 0.7430 Epoch 9/30 60/60 [==============================] - ETA: 0s - loss: 0.8306 - accuracy: 0.7124 Epoch 9: val_accuracy improved from 0.74299 to 0.77804, saving model to best_model.keras 60/60 [==============================] - 15s 244ms/step - loss: 0.8306 - accuracy: 0.7124 - val_loss: 0.7376 - val_accuracy: 0.7780 Epoch 10/30 60/60 [==============================] - ETA: 0s - loss: 0.7548 - accuracy: 0.7412 Epoch 10: val_accuracy improved from 0.77804 to 0.78738, saving model to best_model.keras 60/60 [==============================] - 15s 245ms/step - loss: 0.7548 - accuracy: 0.7412 - val_loss: 0.6998 - val_accuracy: 0.7874 Epoch 11/30 60/60 [==============================] - ETA: 0s - loss: 0.7067 - accuracy: 0.7592 Epoch 11: val_accuracy did not improve from 0.78738 60/60 [==============================] - 15s 244ms/step - loss: 0.7067 - accuracy: 0.7592 - val_loss: 0.6823 - val_accuracy: 0.7850 Epoch 12/30 60/60 [==============================] - ETA: 0s - loss: 0.6505 - accuracy: 0.7782 Epoch 12: val_accuracy did not improve from 0.78738 60/60 [==============================] - 15s 246ms/step - loss: 0.6505 - accuracy: 0.7782 - val_loss: 0.6848 - val_accuracy: 0.7827 Epoch 13/30 60/60 [==============================] - ETA: 0s - loss: 0.6380 - accuracy: 0.7790 Epoch 13: val_accuracy improved from 0.78738 to 0.81075, saving model to best_model.keras 60/60 [==============================] - 15s 246ms/step - loss: 0.6380 - accuracy: 0.7790 - val_loss: 0.6515 - val_accuracy: 0.8107 Epoch 14/30 60/60 [==============================] - ETA: 0s - loss: 0.5984 - accuracy: 0.7851 Epoch 14: val_accuracy improved from 0.81075 to 0.81542, saving model to best_model.keras 60/60 [==============================] - 17s 277ms/step - loss: 0.5984 - accuracy: 0.7851 - val_loss: 0.6358 - val_accuracy: 0.8154 Epoch 15/30 60/60 [==============================] - ETA: 0s - loss: 0.5859 - accuracy: 0.7973 Epoch 15: val_accuracy did not improve from 0.81542 60/60 [==============================] - 15s 245ms/step - loss: 0.5859 - accuracy: 0.7973 - val_loss: 0.6439 - val_accuracy: 0.8061 Epoch 16/30 59/60 [============================>.] - ETA: 0s - loss: 0.5547 - accuracy: 0.8114 Epoch 16: val_accuracy improved from 0.81542 to 0.82243, saving model to best_model.keras 60/60 [==============================] - 15s 244ms/step - loss: 0.5542 - accuracy: 0.8115 - val_loss: 0.5960 - val_accuracy: 0.8224 Epoch 17/30 60/60 [==============================] - ETA: 0s - loss: 0.5314 - accuracy: 0.8094 Epoch 17: val_accuracy improved from 0.82243 to 0.84112, saving model to best_model.keras 60/60 [==============================] - 15s 245ms/step - loss: 0.5314 - accuracy: 0.8094 - val_loss: 0.6013 - val_accuracy: 0.8411 Epoch 18/30 60/60 [==============================] - ETA: 0s - loss: 0.5076 - accuracy: 0.8216 Epoch 18: val_accuracy did not improve from 0.84112 60/60 [==============================] - 15s 245ms/step - loss: 0.5076 - accuracy: 0.8216 - val_loss: 0.6381 - val_accuracy: 0.8201 Epoch 19/30 60/60 [==============================] - ETA: 0s - loss: 0.4838 - accuracy: 0.8308 Epoch 19: val_accuracy did not improve from 0.84112 60/60 [==============================] - 15s 244ms/step - loss: 0.4838 - accuracy: 0.8308 - val_loss: 0.6260 - val_accuracy: 0.8014 Epoch 20/30 60/60 [==============================] - ETA: 0s - loss: 0.4976 - accuracy: 0.8171 Epoch 20: val_accuracy did not improve from 0.84112 60/60 [==============================] - 15s 245ms/step - loss: 0.4976 - accuracy: 0.8171 - val_loss: 0.6103 - val_accuracy: 0.8294 Epoch 21/30 60/60 [==============================] - ETA: 0s - loss: 0.4644 - accuracy: 0.8377 Epoch 21: val_accuracy improved from 0.84112 to 0.84346, saving model to best_model.keras 60/60 [==============================] - 15s 243ms/step - loss: 0.4644 - accuracy: 0.8377 - val_loss: 0.5796 - val_accuracy: 0.8435 Epoch 22/30 60/60 [==============================] - ETA: 0s - loss: 0.4588 - accuracy: 0.8398 Epoch 22: val_accuracy did not improve from 0.84346 60/60 [==============================] - 15s 246ms/step - loss: 0.4588 - accuracy: 0.8398 - val_loss: 0.6117 - val_accuracy: 0.8318 Epoch 23/30 60/60 [==============================] - ETA: 0s - loss: 0.4305 - accuracy: 0.8462 Epoch 23: val_accuracy did not improve from 0.84346 60/60 [==============================] - 15s 242ms/step - loss: 0.4305 - accuracy: 0.8462 - val_loss: 0.5951 - val_accuracy: 0.8318 Epoch 24/30 60/60 [==============================] - ETA: 0s - loss: 0.4339 - accuracy: 0.8472 Epoch 24: val_accuracy did not improve from 0.84346 60/60 [==============================] - 15s 242ms/step - loss: 0.4339 - accuracy: 0.8472 - val_loss: 0.5915 - val_accuracy: 0.8388 Epoch 25/30 60/60 [==============================] - ETA: 0s - loss: 0.3779 - accuracy: 0.8575 Epoch 25: val_accuracy did not improve from 0.84346 60/60 [==============================] - 15s 244ms/step - loss: 0.3779 - accuracy: 0.8575 - val_loss: 0.5772 - val_accuracy: 0.8411 Epoch 26/30 60/60 [==============================] - ETA: 0s - loss: 0.3968 - accuracy: 0.8477 Epoch 26: val_accuracy improved from 0.84346 to 0.85981, saving model to best_model.keras 60/60 [==============================] - 15s 250ms/step - loss: 0.3968 - accuracy: 0.8477 - val_loss: 0.5366 - val_accuracy: 0.8598 Epoch 27/30 60/60 [==============================] - ETA: 0s - loss: 0.3911 - accuracy: 0.8586 Epoch 27: val_accuracy did not improve from 0.85981 60/60 [==============================] - 15s 241ms/step - loss: 0.3911 - accuracy: 0.8586 - val_loss: 0.5985 - val_accuracy: 0.8411 Epoch 28/30 60/60 [==============================] - ETA: 0s - loss: 0.3557 - accuracy: 0.8673 Epoch 28: val_accuracy did not improve from 0.85981 60/60 [==============================] - 14s 241ms/step - loss: 0.3557 - accuracy: 0.8673 - val_loss: 0.6019 - val_accuracy: 0.8435 Epoch 29/30 60/60 [==============================] - ETA: 0s - loss: 0.3605 - accuracy: 0.8662 Epoch 29: val_accuracy did not improve from 0.85981 60/60 [==============================] - 15s 244ms/step - loss: 0.3605 - accuracy: 0.8662 - val_loss: 0.6190 - val_accuracy: 0.8201 Epoch 30/30 60/60 [==============================] - ETA: 0s - loss: 0.3640 - accuracy: 0.8623 Epoch 30: val_accuracy did not improve from 0.85981 60/60 [==============================] - 14s 240ms/step - loss: 0.3640 - accuracy: 0.8623 - val_loss: 0.5569 - val_accuracy: 0.8481
plt.plot(model2_history.history['accuracy'])
plt.plot(model2_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- Accuracy of the model is showing good progress, however Train and Validation data shows converging.
- Model-2 is promising on predection
plt.plot(model2_history.history['loss'])
plt.plot(model2_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
- Loss predection is also reducing but this Model perform better than Model1
Generating the predictions using test data¶
model2_y_pred = model2.predict(X_test_normalized)
model2_y_pred_classes_ln = np.argmax(model2_y_pred,axis=1)
model2_y_test = np.argmax(y_test_encoded, axis=1)
15/15 [==============================] - 0s 7ms/step
# Test Accuracy for Model2
accuracy_score(model2_y_test, model2_y_pred_classes_ln)
0.8357894736842105
- There is 84% accruacy in Model-2 and performing better than Model-1
Plotting Confusion Matrix¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(model2_y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Observation:
- The confusion matrix appears to be really good.
- The 6th class is the most confused class among all and class 0 still shows misclassification
- We can observe that this model has performed well as compared to previous model.
Model- 3 (Transfer Learning using VGG16)¶
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
Let's try again, but this time, we will be using the idea of Transfer Learning. We will be loading a pre-built architecture - VGG16, which was trained on the ImageNet dataset and is the runner-up in the ImageNet competition in 2014.
For training VGG16, we will directly use the convolutional and pooling layers and freeze their weights i.e. no training will be done on them. For classification, we will replace the existing fully-connected layers with FC layers created specifically for our problem.
from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (128,128,3))
vgg_model.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 128, 128, 3)] 0
block1_conv1 (Conv2D) (None, 128, 128, 64) 1792
block1_conv2 (Conv2D) (None, 128, 128, 64) 36928
block1_pool (MaxPooling2D) (None, 64, 64, 64) 0
block2_conv1 (Conv2D) (None, 64, 64, 128) 73856
block2_conv2 (Conv2D) (None, 64, 64, 128) 147584
block2_pool (MaxPooling2D) (None, 32, 32, 128) 0
block3_conv1 (Conv2D) (None, 32, 32, 256) 295168
block3_conv2 (Conv2D) (None, 32, 32, 256) 590080
block3_conv3 (Conv2D) (None, 32, 32, 256) 590080
block3_pool (MaxPooling2D) (None, 16, 16, 256) 0
block4_conv1 (Conv2D) (None, 16, 16, 512) 1180160
block4_conv2 (Conv2D) (None, 16, 16, 512) 2359808
block4_conv3 (Conv2D) (None, 16, 16, 512) 2359808
block4_pool (MaxPooling2D) (None, 8, 8, 512) 0
block5_conv1 (Conv2D) (None, 8, 8, 512) 2359808
block5_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block5_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block5_pool (MaxPooling2D) (None, 4, 4, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
layer.trainable = False
for layer in vgg_model.layers:
print(layer.name, layer.trainable)
input_1 False block1_conv1 False block1_conv2 False block1_pool False block2_conv1 False block2_conv2 False block2_pool False block3_conv1 False block3_conv2 False block3_conv3 False block3_pool False block4_conv1 False block4_conv2 False block4_conv3 False block4_pool False block5_conv1 False block5_conv2 False block5_conv3 False block5_pool False
model3 = Sequential()
# Adding the convolutional part of the VGG16 model from above
model3.add(vgg_model)
# Flattening the output of the VGG16 model because it is from a convolutional layer
model3.add(Flatten())
# Adding a dense output layer with 16 neurons
model3.add(Dense(100, activation='relu'))
model3.add(Dense(50, activation='relu'))
model3.add(Dropout(0.3))
model3.add(Dense(12, activation='softmax'))
opt=Adam()
# Compile model
model3.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model3.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 4, 4, 512) 14714688
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 100) 819300
dense_1 (Dense) (None, 50) 5050
dropout (Dropout) (None, 50) 0
dense_2 (Dense) (None, 12) 612
=================================================================
Total params: 15539650 (59.28 MB)
Trainable params: 824962 (3.15 MB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
# Epochs
epochs = 30
# Batch size
batch_size = 64
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.keras', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model3_vgg16_history = model3.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1, callbacks=[es, mc])
Epoch 1/30 60/60 [==============================] - ETA: 0s - loss: 2.2362 - accuracy: 0.2255 Epoch 1: val_accuracy improved from -inf to 0.35280, saving model to best_model.keras 60/60 [==============================] - 20s 296ms/step - loss: 2.2362 - accuracy: 0.2255 - val_loss: 1.9320 - val_accuracy: 0.3528 Epoch 2/30 60/60 [==============================] - ETA: 0s - loss: 1.8613 - accuracy: 0.3695 Epoch 2: val_accuracy improved from 0.35280 to 0.46495, saving model to best_model.keras 60/60 [==============================] - 16s 271ms/step - loss: 1.8613 - accuracy: 0.3695 - val_loss: 1.6281 - val_accuracy: 0.4650 Epoch 3/30 60/60 [==============================] - ETA: 0s - loss: 1.5698 - accuracy: 0.4592 Epoch 3: val_accuracy improved from 0.46495 to 0.50234, saving model to best_model.keras 60/60 [==============================] - 16s 272ms/step - loss: 1.5698 - accuracy: 0.4592 - val_loss: 1.4620 - val_accuracy: 0.5023 Epoch 4/30 60/60 [==============================] - ETA: 0s - loss: 1.3894 - accuracy: 0.5247 Epoch 4: val_accuracy improved from 0.50234 to 0.55607, saving model to best_model.keras 60/60 [==============================] - 16s 270ms/step - loss: 1.3894 - accuracy: 0.5247 - val_loss: 1.3201 - val_accuracy: 0.5561 Epoch 5/30 60/60 [==============================] - ETA: 0s - loss: 1.2598 - accuracy: 0.5586 Epoch 5: val_accuracy improved from 0.55607 to 0.58645, saving model to best_model.keras 60/60 [==============================] - 17s 278ms/step - loss: 1.2598 - accuracy: 0.5586 - val_loss: 1.1780 - val_accuracy: 0.5864 Epoch 6/30 60/60 [==============================] - ETA: 0s - loss: 1.1506 - accuracy: 0.5998 Epoch 6: val_accuracy improved from 0.58645 to 0.61215, saving model to best_model.keras 60/60 [==============================] - 17s 281ms/step - loss: 1.1506 - accuracy: 0.5998 - val_loss: 1.1073 - val_accuracy: 0.6121 Epoch 7/30 60/60 [==============================] - ETA: 0s - loss: 1.0675 - accuracy: 0.6389 Epoch 7: val_accuracy improved from 0.61215 to 0.64720, saving model to best_model.keras 60/60 [==============================] - 16s 271ms/step - loss: 1.0675 - accuracy: 0.6389 - val_loss: 1.0602 - val_accuracy: 0.6472 Epoch 8/30 60/60 [==============================] - ETA: 0s - loss: 0.9709 - accuracy: 0.6646 Epoch 8: val_accuracy improved from 0.64720 to 0.65654, saving model to best_model.keras 60/60 [==============================] - 17s 279ms/step - loss: 0.9709 - accuracy: 0.6646 - val_loss: 0.9959 - val_accuracy: 0.6565 Epoch 9/30 60/60 [==============================] - ETA: 0s - loss: 0.9037 - accuracy: 0.6931 Epoch 9: val_accuracy improved from 0.65654 to 0.68692, saving model to best_model.keras 60/60 [==============================] - 16s 270ms/step - loss: 0.9037 - accuracy: 0.6931 - val_loss: 0.9735 - val_accuracy: 0.6869 Epoch 10/30 60/60 [==============================] - ETA: 0s - loss: 0.8738 - accuracy: 0.6973 Epoch 10: val_accuracy did not improve from 0.68692 60/60 [==============================] - 16s 268ms/step - loss: 0.8738 - accuracy: 0.6973 - val_loss: 1.0857 - val_accuracy: 0.6145 Epoch 11/30 60/60 [==============================] - ETA: 0s - loss: 0.8258 - accuracy: 0.7209 Epoch 11: val_accuracy did not improve from 0.68692 60/60 [==============================] - 16s 265ms/step - loss: 0.8258 - accuracy: 0.7209 - val_loss: 0.9379 - val_accuracy: 0.6612 Epoch 12/30 60/60 [==============================] - ETA: 0s - loss: 0.7404 - accuracy: 0.7473 Epoch 12: val_accuracy improved from 0.68692 to 0.68925, saving model to best_model.keras 60/60 [==============================] - 16s 271ms/step - loss: 0.7404 - accuracy: 0.7473 - val_loss: 0.8890 - val_accuracy: 0.6893 Epoch 13/30 60/60 [==============================] - ETA: 0s - loss: 0.7500 - accuracy: 0.7454 Epoch 13: val_accuracy did not improve from 0.68925 60/60 [==============================] - 16s 263ms/step - loss: 0.7500 - accuracy: 0.7454 - val_loss: 0.9844 - val_accuracy: 0.6636 Epoch 14/30 60/60 [==============================] - ETA: 0s - loss: 0.7248 - accuracy: 0.7462 Epoch 14: val_accuracy improved from 0.68925 to 0.69860, saving model to best_model.keras 60/60 [==============================] - 16s 273ms/step - loss: 0.7248 - accuracy: 0.7462 - val_loss: 0.8616 - val_accuracy: 0.6986 Epoch 15/30 60/60 [==============================] - ETA: 0s - loss: 0.6695 - accuracy: 0.7666 Epoch 15: val_accuracy improved from 0.69860 to 0.70794, saving model to best_model.keras 60/60 [==============================] - 16s 272ms/step - loss: 0.6695 - accuracy: 0.7666 - val_loss: 0.8572 - val_accuracy: 0.7079 Epoch 16/30 59/60 [============================>.] - ETA: 0s - loss: 0.6434 - accuracy: 0.7820 Epoch 16: val_accuracy did not improve from 0.70794 60/60 [==============================] - 17s 278ms/step - loss: 0.6431 - accuracy: 0.7822 - val_loss: 0.9066 - val_accuracy: 0.6729 Epoch 17/30 60/60 [==============================] - ETA: 0s - loss: 0.6526 - accuracy: 0.7793 Epoch 17: val_accuracy improved from 0.70794 to 0.72897, saving model to best_model.keras 60/60 [==============================] - 16s 271ms/step - loss: 0.6526 - accuracy: 0.7793 - val_loss: 0.8180 - val_accuracy: 0.7290 Epoch 18/30 60/60 [==============================] - ETA: 0s - loss: 0.5887 - accuracy: 0.8020 Epoch 18: val_accuracy did not improve from 0.72897 60/60 [==============================] - 16s 263ms/step - loss: 0.5887 - accuracy: 0.8020 - val_loss: 0.8798 - val_accuracy: 0.6963 Epoch 19/30 60/60 [==============================] - ETA: 0s - loss: 0.5763 - accuracy: 0.8049 Epoch 19: val_accuracy did not improve from 0.72897 60/60 [==============================] - 16s 263ms/step - loss: 0.5763 - accuracy: 0.8049 - val_loss: 0.8508 - val_accuracy: 0.7079 Epoch 20/30 60/60 [==============================] - ETA: 0s - loss: 0.5802 - accuracy: 0.8002 Epoch 20: val_accuracy did not improve from 0.72897 60/60 [==============================] - 16s 264ms/step - loss: 0.5802 - accuracy: 0.8002 - val_loss: 0.8318 - val_accuracy: 0.7266 Epoch 21/30 60/60 [==============================] - ETA: 0s - loss: 0.5227 - accuracy: 0.8245 Epoch 21: val_accuracy did not improve from 0.72897 60/60 [==============================] - 16s 263ms/step - loss: 0.5227 - accuracy: 0.8245 - val_loss: 0.8510 - val_accuracy: 0.7079 Epoch 22/30 60/60 [==============================] - ETA: 0s - loss: 0.5091 - accuracy: 0.8332 Epoch 22: val_accuracy did not improve from 0.72897 60/60 [==============================] - 16s 268ms/step - loss: 0.5091 - accuracy: 0.8332 - val_loss: 0.8277 - val_accuracy: 0.7220 Epoch 22: early stopping
plt.plot(model3_vgg16_history.history['accuracy'])
plt.plot(model3_vgg16_history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
plt.plot(model3_vgg16_history.history['loss'])
plt.plot(model3_vgg16_history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Generating the predictions using test data¶
# Here we would get the output as probablities for each category
model3_y_pred = model3.predict(X_test_normalized)
model3_y_pred_classes_ln = np.argmax(model3_y_pred,axis=1)
model3_y_test = np.argmax(y_test_encoded, axis=1)
15/15 [==============================] - 1s 45ms/step
# Test Accuracy Model3
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(model3_y_test, model3_y_pred_classes_ln)
0.72
- Model-3 performed poor as compared to Model-1 and 2.
- Accuracy rate is just 72%
Plotting Confusion Matrix¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(model3_y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
- Confusion matricx indicates that, this model is not able to predict for lables 6 and 8 properly
- But predicts something else.
Classification Report for each class¶
Precision: precision is the fraction of relevant instances among the retrieved instances.
Recall: recall is the fraction of relevant instances that were retrieved.
F1 score: The F1 score is the harmonic mean of precision and recall, reaching its optimal value at 1 and its worst value at 0.
For ALL Models¶
print("Model 1")
print(classification_report(model1_y_test, model1_y_pred_classes_ln))
Model 1
precision recall f1-score support
0 0.46 0.23 0.31 26
1 0.83 0.97 0.89 39
2 0.80 0.83 0.81 29
3 0.86 0.84 0.85 61
4 0.53 0.77 0.63 22
5 0.77 0.71 0.74 48
6 0.71 0.82 0.76 65
7 0.76 0.59 0.67 22
8 0.72 0.79 0.75 52
9 0.80 0.70 0.74 23
10 0.98 0.82 0.89 50
11 0.80 0.84 0.82 38
accuracy 0.77 475
macro avg 0.75 0.74 0.74 475
weighted avg 0.77 0.77 0.77 475
print("Model 2")
print(classification_report(model2_y_test, model2_y_pred_classes_ln))
Model 2
precision recall f1-score support
0 0.55 0.42 0.48 26
1 0.97 0.90 0.93 39
2 0.80 0.69 0.74 29
3 0.90 0.93 0.92 61
4 0.78 0.82 0.80 22
5 0.93 0.90 0.91 48
6 0.77 0.85 0.81 65
7 0.86 0.82 0.84 22
8 0.76 0.90 0.82 52
9 0.76 0.57 0.65 23
10 0.90 0.92 0.91 50
11 0.85 0.89 0.87 38
accuracy 0.84 475
macro avg 0.82 0.80 0.81 475
weighted avg 0.83 0.84 0.83 475
print("Model 3")
print(classification_report(model3_y_test, model3_y_pred_classes_ln))
Model 3
precision recall f1-score support
0 0.53 0.38 0.44 26
1 0.66 0.69 0.68 39
2 0.80 0.69 0.74 29
3 0.84 0.77 0.80 61
4 0.55 0.55 0.55 22
5 0.85 0.48 0.61 48
6 0.68 0.83 0.74 65
7 0.70 0.73 0.71 22
8 0.65 0.85 0.73 52
9 0.79 0.48 0.59 23
10 0.82 0.90 0.86 50
11 0.73 0.87 0.80 38
accuracy 0.72 475
macro avg 0.71 0.68 0.69 475
weighted avg 0.73 0.72 0.71 475
Final Model¶
- Model-2 performed really well as compared to other model.
- We can run a test to visualize how each model predection works
Model comparison¶
Visualizing the prediction¶
def visualize_predicted_images(model, images, predicted_labels, actual_labels):
num_classes = 10 # Number of Classes
# Get predictions for the predicted_labels (images)
predicted_labels_decoded = enc.inverse_transform(model.predict(predicted_labels))
predicted_keys = dict(enumerate(predicted_labels_decoded)) # Create a dictionary from predicted labels
# Get the actual labels (not predictions)
actual_labels_decoded = enc.inverse_transform(actual_labels) # Use actual_labels directly
actual_keys = dict(enumerate(actual_labels_decoded)) # Create a dictionary from actual labels
rows = 3 # Defining number of rows=3
cols = 4 # Defining number of columns=4
fig = plt.figure(figsize=(40, 20)) # Defining the figure size to 10x8
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(predicted_labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
ax.imshow(images[random_index, :]) # Plotting the image
ax.set_ylabel("Actual :" + actual_keys[random_index]) # Use actual label here
ax.set_xlabel("Predicted : " + predicted_keys[random_index])
plt.show()
Model-1 Predection¶
visualize_predicted_images(model1, X_test, X_test_normalized, y_test_encoded)
15/15 [==============================] - 0s 7ms/step
Model-2 Predection¶
visualize_predicted_images(model2, X_test, X_test_normalized, y_test_encoded)
15/15 [==============================] - 0s 7ms/step
Model-3 Predection¶
visualize_predicted_images(model3, X_test, X_test_normalized, y_test_encoded)
15/15 [==============================] - 1s 46ms/step
Actionable Insights and Business Recommendations¶
- As we have seen, the CNN model - 2 was able to predict the test image correctly with a test accuracy of 84%. Model-1 came to 77% and Model-3 with 72%
- All the Models can be further fine tuned to achieve higher accuracy by adding more dense layer.
- Early Stopping greatly helped all the model on accuracy, with out Early Stopping Moldes were performing poorly
- I will stop here at 84% accuracy due to availabilitty of resources.
- I used decreased image size for initial model building, but the predection on modified image size was below 60%. So I went ahead with origional image to create Train, Validation and Test data
- Above models will provide better accuracy with higher resoulution images but processing it will require better infrastructure.
%%shell
jupyter nbconvert --to html '/content/drive/MyDrive/AIMLColab/Intro to Computer Vision/CNN-Project1/CV_Project_Full_Code_Notebook.ipynb'
[NbConvertApp] Converting notebook /content/drive/MyDrive/AIMLColab/Intro to Computer Vision/CNN-Project1/CV_Project_Full_Code_Notebook.ipynb to html [NbConvertApp] WARNING | Alternative text is missing on 18 image(s). [NbConvertApp] Writing 5078523 bytes to /content/drive/MyDrive/AIMLColab/Intro to Computer Vision/CNN-Project1/CV_Project_Full_Code_Notebook.html